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Adequately  testing,  evaluating  and  analyzing  large,  scale-free  information  systems  is  very 
difficult  due  to  the  nebulous  nature  of  the  systems  boundaries  and  ever-changing  number 
of  links  and  nodes  that  comprise  the  overall  system.  This  paper  proposed  that  it  is 
possible  to  conduct  such  studies  through  distributed  holistic  bounding  and  employment  of 
disciplined  experiment  campaigning.  Beginning  with  identification  of  the  overarching 
purpose  of  the  entire  system  and  identifying  the  boundaries  of  the  sub-systems  that 
contribute  to  the  overall  system’s  functionality  we  can  develop  effective  experiments  that 
will  identify  critical  environmental  variables  that  affect  system  performance.  By 
adhering  to  an  experiment  campaign  strategy  we  can  discover  those  variables,  develop 
hypotheses  that  help  determine  optimal  system  configurations  and  ultimately  build  and 
demonstrate  that  optimal  system. 

Introduction 


The  United  States  Department  of  Defense  (DoD)  is  a  global  organization  that  relies  heavily 
on  robust,  reliable  information  systems  to  communicate  around  the  world.  Without  adequate 
communication  systems  the  DoD  might  not  be  able  to  meet  its  national  security  obligations 
and  the  security  of  the  United  States  could  be  placed  in  jeopardy.  “Information  Superiority” 
is  the  term  DoD  uses  to  refer  to  maintaining  communication  paths  and  providing  its  forces 
with  complete  and  accurate  data,  above  and  beyond  that  of  any  enemy.  To  meet  the  extensive 
communication  and  data  intensive  requirements  needed  to  achieve  Information  Superiority 
the  DoD  has  developed  the  Global  Information  Grid  (GIG).  (US  Department  of  Defense 
2002).  The  GIG  is  a  combination  of  systems  including  artifact  systems,  human  systems 
(organizational  and  psychological),  and  hybrid  systems  dealing  with  human-machine 
interfaces  that  provide  information  services  to  globally  deployed  military  decision-makers. 
The  ubiquitous  nature  of  the  GIG  means  the  overall  network  will  consist  of  constantly 
changing  numbers  of  nodes  and  links.  This  type  of  network  is  considered  to  be  a  scale-free 
system  (Baribasi  2003).  The  ever-changing  boundaries  of  a  scale-free  system  make  it 
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difficult  to  test  and  evaluate  overall  system  performance,  but  it  is  not  only  the  scale-free 
nature  of  the  GIG  that  makes  testing  and  evaluation  of  the  complete  system  difficult,  the 
complexity  associated  the  nebulous  and  complex  nature  of  its  purpose  (i.e.,  achieving 
Information  Superiority)  also  contribute  to  difficulties  in  bounding  any  experiments 
associated  with  the  GIG. 

Information  Superiority,  by  definition,  is  a  measure  of  one  force’s  information  capabilities 
relative  to  those  of  an  opponent.  It  would  be  impractical,  however,  to  evaluate  systems  within 
the  GIG  against  every  possible  opponent.  Therefore  metrics  associated  with  GIG  systems 
must  be  identified  that  can  be  measured  exclusive  of  a  specific  opponent  yet  still  produce 
results  that  indicate  probabilities  of  reaching  Information  Superiority  (the  ultimate  purpose  of 
GIG  systems.)  Although  there  are  no  established  (i.e.,  universally  accepted)  criteria  for 
measuring  the  effectiveness  of  large  information  systems,  there  are  three  data  metrics  that 
can  be  used  to  evaluate  the  quality  of  information  associated  with  achieving  Information 
Superiority:  completeness ,  correctness  and  currency  (Perry,  Signori,  Boon  2004). 

Simply  measuring  the  completeness,  correctness  or  currency  of  information  within  elements 
of  the  GIG  is  not  enough  to  aid  in  achieving  Information  Superiority.  If  any  examination 
should  show  a  deficiency  in  one  of  these  areas  there  must  be  a  feedback  mechanism  in  place 
that  will  help  identify  and  correct  the  discrepancy.  The  scale-free  nature  of  the  GIG  and  its 
large  footprint  (i.e.,  global  deployment)  make  development  and  measuring  of  specific 
feedback  mechanisms  a  daunting  task  indeed.  In  this  paper  we  will  demonstrate  that  this  can 
be  accomplished  through  careful  bounding  of  individual  GIG  systems  within  the  context  of 
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the  overall  objective.  We  will  explore  the  significance  of  identifying  system  boundaries  and 
parameters  associated  with  specific  sub-systems  within  the  larger  network.  We  will  also 
provide  suggested  methods  for  locating  these  system  boundaries  and  developing  a  listing  of 
parameters  necessary  to  adequately  assess  specific  performance  characteristics  of  the  system. 
Through  this  examination  of  GIG  feedback  mechanisms  we  will  provide  a  methodology  for 
developing  experiment  designs  that  can  be  applied  to  any  large,  scale-free  network. 

The  role  of  feedback  in  information  systems 

Generally,  machines  and  electronic  systems,  if  left  unattended  or  not  maintained  will  strive  to 
achieve  their  lowest  energy  state  (i.e.,  they  will  stop  functioning.)  This  is  an  example  of  open 
loop  control.  Open  loop  control  systems  use  no  measurement  of  outputs  and  no 
compensation  for  deviations  from  desired  outcomes  (Franklin  1994).  Therefore,  keeping  a 
system  operating  effectively  (i.e.,  in  a  manner  that  allows  the  system  to  achieve  its 
objectives)  requires  some  intervention.  This  is  found  in  closed  loop  systems  and  is  identified 
by  the  existence  of  a  feedback  loop.  This  intervention  (feedback)  can  be  accomplished 
manually  by  humans  or  automated  within  the  system  or  a  combination  of  the  two. 

Regardless  of  the  method,  feedback  is  its  own  process  within  the  system  itself  and  must  be 
accounted  for  in  system  design  and  implementation. 

To  better  understand  the  feedback  process  (as  we  will  apply  it  to  the  GIG)  there  are  some 
definitional  conventions  that  need  to  be  established.  Feedback  is  a  method  of  controlling  a 
system.  Control  is  the  process  of  causing  a  chosen  system  variable  to  conform  to  a  desired 
value.  Feedback  is  the  process  of  measuring  the  controlled  variable  and  using  that 
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information  to  influence  the  value  of  the  controlled  variable  (Franklin  1994).  The  significant 
aspect  of  feedback  within  a  system  is  that  it  should  be  integral  to  the  system’s  design  (i.e.,  it 
should  not  be  an  ad-hoc  application  of  actions  based  on  random  measurements  after  the 
system  has  been  fielded.)  Feedback  mechanisms  that  may  be  used  to  control  large,  scale-free 
systems  may  become  quite  complex;  regardless  of  the  complexity  of  feedback  systems  they 
all  contain  the  same  basic  elements: 

•  A  reference  (desired  outcome) 

•  Output  sensors  (to  measure  actual  outcome) 

•  A  comparator  (which  compares  output  data  with  input  data) 

•  A  process  (for  doing  useful  work  with  the  compared  data),  and 

•  An  actuator  (which  influences  the  input  variable.) 

For  example,  an  oven  may  be  designed  to  provide  constant  heat  in  its  chamber.  To  do  this  it 
must  1)  “know”  the  desired  temperature  (reference)  2)  “know”  its  current  temperature 
(through  an  output  sensor),  3)  have  a  means  of  determining  the  difference  between  (1)  and 
(2)  (comparator)  4)  have  a  means  of  transmitting  that  data  to  the  system  so  it  can  make 
adjustments  to  eliminate  the  difference  between  (1)  and  (2)  (a  process)  and  5)  a  means  of 
adjusting  the  heating  element  to  reach  the  reference  temperature  (actuator.)  The  feedback 
system  is  using  data  from  the  environment  (temperature)  to  control  the  output  of  the  heating 
mechanism  (actuator)  to  meet  its  objective  (proper  temperature  in  the  oven’s  chamber.) 

This  is  an  example  of  a  very  simple  automated  control  process  using  a  feedback  system.  As 
mentioned,  the  GIG  is  considerably  more  complex  and  provides  some  interesting  challenges 
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in  ensuring  effective  system  performance  but  by  focusing  on  the  fundamental  elements  of  a 
basic  feedback  mechanism  we  can  more  adequately  evaluate  the  effectiveness  of  the 
mechanism  within  the  system  under  study. 

As  we  can  see  from  the  definition  of  feedback  it  is  essential  to  identify  the  variables  that 
most  affect  system  output  and  that  may  be  controlled  or  overcome  by  the  system.  In  this 
study  there  are  many  environmental  and  physical  system  variables  that  may  contribute  to 
data  loss.  A  system  designed  to  achieve  one  objective  (e.g.,  temperature  control  in  a  single 
chamber)  may  require  only  one  sensor  and  one  process  to  control  the  outcome.  A  more 
complex  system,  such  as  the  GIG,  may  require  more  control  and  feedback  processes  to  reach 
a  desired  outcome;  more  complexity  will  occur  if  these  controlled  variables  are  inputs, 
outputs  or  somehow  affect  other  processes  in  the  system. 

The  relationship  between  adaptability  and  feedback 

As  we  will  explore  later,  establishing  the  boundaries  of  a  system  is  essential  to  studying  and 
understanding  the  functions  of  that  system.  This  section  will  explore  the  areas  of  most 
concern  to  the  system  under  study  and  will  help  identify  specific  criteria  for  measurement. 
Identifying  these  criteria  will  be  essential  to  the  experiment  we  will  outline  at  the  end  of  this 
paper. 

Adaptation  is  the  first  system  characteristic  we  will  explore  in  relation  to  system  viability, 
survivability  and  utility.  Adaptation  occurs  when  an  entity  undergoes  some  change  to  fit  a 
new  or  special  situation.  In  our  case,  these  new  or  special  situations  are  the  changing 
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environment  in  which  the  system  is  operating.  There  are  two  forms  of  adaptation  that  should 
be  considered  when  looking  at  a  system.  One  is  the  long-term  adaptation  where  the  system 
changes  over  an  extended  period  of  time  in  reaction  to  technological  changes,  or  political  and 
legal  changes.  This  form  of  adaptation  may  be  seen  as  following  evolutionary  theories  where 
individual  organisms  do  not  specifically  adapt  to  an  environmental  change  but  rather  slow 
changes  over  time  ensure  the  survival  of  the  species  (i.e.,  system)  (Platt  1970).  The  second 
form  is  short-term  adaptation  where  the  system  (i.e.,  individual  organism)  reacts  to 
immediate  changes  in  the  environment  to  ensure  its  own  survival  (Chakravarthy  1982).  Our 
focus  will  be  on  the  latter  form  of  adaptation. 

In  testing  the  ability  of  an  information  system  to  adapt  to  changing  conditions  we  must 
identify  the  environmental  factors  that  may  change  in  such  a  way  that  the  system  may  not 
meet  its  operational  objective.  We  must  then  determine  if  these  apply  to  our  metrics  of 
completeness,  correctness  or  currency.  For  the  preliminary  experiment  we  will  look  at  the 
major  contributors  to  potential  data  loss  during  transmission  or  reception  (i.e.,  completeness.) 
At  this  point  we  will  control  the  initial  data  (i.e.,  correctness)  to  ensure  there  is  a  good  signal 
at  the  input  to  better  measure  the  adaptability  of  the  system  to  a  changing  physical 
environment1. 

The  experimental  campaign  for  this  system  should  take  into  account  all  aspects  of  the  system 
and  the  environment  that  might  contribute  to  data  losses  between  the  source  and  the  receiver. 

’Since  our  experiment  will  focus  on  the  physical  characteristics  of  the  information  system  we  will  not  address 

human  elements  of  adaptability  that  may  affect  the  system  (e.g.,  user  expertise  and  knowledge,  etc.) 
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The  following  six  sources  of  signal  loss/degradation  that  exist  in  this  system’s  environment2 
(Sklarl988) 

•  Pointing  loss 

•  Atmospheric  loss  and  noise 

•  Space  loss 

•  Adjacent  channel  interference 

•  Co-channel  interference 

•  Galactic  or  cosmic,  star,  and  terrestrial  noise 

The  parameters  for  evaluating  this  system  should  be  based  on  these  elements  and  their 
expected  prevalence  and  influence  in  the  expected  operating  environment.  The  system  under 
study  will  be  operating  in  dynamic  meteorological  and  geographic  conditions  and  must 
demonstrate  an  ability  to  overcome  any  adverse  environmental  affects  to  be  reach  its  objects. 
In  this  study  it  will  be  the  human  receiver  that  will  determine  if  there  is  significant  data  loss. 
This  information  must  be  fed  back  to  the  source  to  allow  retransmission  or  system  alteration 
to  compensate  (i.e.,  adapt)  to  the  changed  environment.  Currency,  in  the  form  of  information 
timeliness,  will  also  be  investigated  during  the  course  of  the  experiment. 

Bounding  the  Problem:  Parsing  the  GIG 

The  United  States  military  operates  around  the  world  and  must  therefore  have  efficient  and 
reliable  global  communications  capability.  Although  the  US  Government  had  been 
experimenting  with  global  satellite  communication  through  the  Advanced  Research  Projects 

2  Sklar  provides  a  more  comprehensive  listing  of  factors  affecting  signal  loss,  the  items  listed  here  are  considered  to 
be  those  that  will  be  most  prevalent  in  the  system’s  operating  environment. 
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Agency  (ARP A)  in  the  late  1950s,  it  was  not  until  1961  that  the  US  Joint  Chiefs  of  Staff 
validated  a  requirement  to  incorporate  satellite  communications  capability  into  its  Defense 
Communications  System  (DCS)  (Pachall  1984).  The  DCS  was  a  communication  system 
with  a  global  reach.  The  Defense  Communications  Agency  (DCA)  was  established  to 
oversee  and  manage  the  DCS  in  1960  and  later  became  the  Defense  Information  Systems 
Agency  (DISA);  the  DCS  was  then  renamed  the  Defense  Information  Infrastructure  (DII) 
through  Defense  Management  Report  Decision  (DMRD)  918.  The  DII  is  now  known  as  the 
Global  Information  Grid  (GIG).  (U.S.  Defense  Information  Systems  Agency  2005) 

The  GIG  qualifies  as  a  scale-free  network  in  that  it  is  considered  to  be  the  globally 
interconnected,  end-to-end  set  of  information  capabilities,  associated  processes,  and 
personnel  for  collecting,  processing,  storing,  disseminating  and  managing  information  on 
demand  to  warfighters,  policy  makers,  and  support  personnel.  The  GIG  includes  all  owned 
and  leased  communications  and  computing  services,  software  (including  applications),  data, 
security  services,  and  other  associated  services  necessary  to  achieve  Information  Superiority 
(U.S.  Department  of  Defense  2002). 

The  strengths  of  this  description  are  that  it  identifies  the  components  of  the  GIG  and  provides 
a  means  of  determining  what  organizations  and  personnel  are  associated  with  the  system.  It 
also  identifies  an  objective  for  the  system:  achieving  Information  Superiority.  The 
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weaknesses  of  this  description  are  that  very  few  organizations  and  personnel  fall  outside  of 
the  system  and  the  objective  is  immeasurable3. 

As  discussed  earlier,  Information  Superiority  is  a  relative  measure;  it  is  the  measure  of  one’s 
own  capabilities  against  those  of  a  specific  opponent.  By  definition  it  is  the  capability  to 
collect,  process,  and  disseminate  an  uninterrupted  flow  of  information  while  exploiting  or 
denying  an  adversary's  ability  to  do  the  same.  Unfortunately  this  definition  does  not  address 
the  quality  of  information,  it  merely  acknowledges  information  flows.  Rephrasing  the 
definition  of  the  GIG  to  incorporate  the  definition  of  Information  Superiority  (the  objective 
of  the  GIG)  gives  us:  The  GIG  is  the  globally  interconnected  set  of  all  machines  and  all 
humans  associated  with  any  information  associated  with  military  operations  the  purpose  of 
which  is  to  ensure  that  there  is  always  a  flow  of  information. 

The  existence  of  many  types  of  systems  and  personnel  within  the  GIG  render  it  nearly  (if  not 
completely)  impossible  to  identify  and  account  for  all  the  variables  that  could  affect  overall 
system  performance.  One  approach  to  developing  effective  feedback  mechanisms  that  will 
ensure  reliability  of  the  GIG  may  be  to  distribute  the  functionality  across  systems. 

This  approach  would  view  the  GIG  as  a  single  open-system  operating  in  the  global 
battlespace.  The  output  sensors  (the  operational  commander  responsible  for  military 
activities  within  the  battlespace)  would  monitor  the  GIG  performance  to  determine  if  it 

3  Although  there  are  proposed  methods  for  measuring  information  utility  and  value  (Perry,  Signori  and  Boon  2004) 
DoD  has  established  no  specific  metrics  to  measure  Information  Superiority  itself. 


9 


S.JIatrou 


helped  achieve  Information  Superiority.  If  the  commander  were  to  be  satisfied  with  the 
performance  of  the  GIG  there  would  be  no  specific  direction  for  alteration  of  inputs  (positive 
feedback).  If,  however,  the  commander  determined  that  the  Information  Superiority  being 
delivered  by  the  GIG,  he/she  would  be  able  to  activate  a  sequence  of  events  within  the  GIG 
that  would  correct  inputs  or  processes  to  bring  the  output  (Information  Superiority)  in  line 
with  expected/desired  outcomes  (negative  feedback).  The  feedback  process  that  would  ensue 
from  this  point  will  be  elaborate  and  complex  but  can  still  be  effective  and  controllable  if  the 
systems  and  processes  that  comprise  the  GIG  are  understood  holistically  in  their  own  right. 

One  possible  means  of  accomplishing  this  is  to  adapt  a  combined  methodology  of 
stakeholder  influences  (Rowley  1997)  and  network  cliques  (Provan  and  Sebastian  1998). 
Rowley  explored  the  significant  impact  that  can  be  made  on  network  performance  by 
constituents  using  the  network  at  various  levels  within  the  organization;  Provan  and 
Sebastian  explored  the  issues  of  attempting  to  integrate  full  networks  across  independent 
organizations  involved  in  the  same  functional  area  (i.e.,  multi-firm  networks.)  By 
combining  these  two  areas  we  may  be  able  to  more  effectively  isolate  the  logical  boundaries 
that  identify  influences  across  ‘nodes’  in  the  GIG  thereby  allowing  the  feedback  process  to 
correctly  adjust  the  appropriate  input  to  effect  necessary  corrections  in  the  GIG  process. 

The  initial  purpose  of  this  paper  was  to  identify  appropriate  methods  for  developing  useful 
feedback  processes  for  the  GIG.  Through  holistic  treatment  of  the  GIG  we  can  identify 
shortcomings  in  its  outputs  that  will  trigger  the  negative  feedback  process.  This  process 
would  involve  a  holistic  view  of  the  ‘cliques’  that  make  up  the  GIG  and  identify  anomalous 
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outputs  that  would,  in  turn,  activate  negative  feedback  processes  within  those  cliques  to 
adjust  system  functions  to  regain  Information  Superiority.  Further  exploration  of  this 
concept  will  be  necessary  to  develop  methods  of  identifying  cliques  within  the  GIG  and  the 
applicability  of  stakeholder  theory  to  this  process. 

Rather  than  attempt  to  examine  the  entire  GIG  and  identify  feedback  mechanisms  that  might 
control  its  outputs,  we  will  explore  one  subsystem  that  interacts  with,  or  is  a  part  of,  the  GIG. 
This  exploration  should  help  identify  the  means  through  which  future  evaluations  of 
feedback  within  the  GIG  can  be  made. 

Selecting  a  subsystem  (clique)  for  study  involved  identification  of  a  significant  information 
system  within  the  GIG  that  performed  a  specific  function  (i.e.,  produced  a  measurable 
outcome.)  The  sub-system  chosen  for  this  study  was  the  Lawrence  Livermore  National 
Laboratory/Tactical  Satellite  (LLNL/TacSAT)  sub-system  of  the  Tactical  Network  Topology 
(TNT). 

The  first  step  in  adequately  bounding  the  system  for  study  is  to  identify  the  specific  objective 
or  outcome  and  its  associated  stakeholders.  Building  on  concepts  of  clique  identification 
(Poran  and  Sabastien  1998)  and  stakeholder  influences  (Rowley  1997)  we  will  be  able  to 
identify  the  systems  and  stakeholders  pertinent  to  our  study.  This  is  accomplished  by 
examining  the  elements  that  affect  the  desired  outcome  and  their  relationships  to  each  other. 
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Identifying  the  elements  begins  with  an  assessment  of  the  LLNL/TacSAT  system  and  its 
contributions  to  Information  Superiority.  In  identifying  significant  elements  it  is  useful  to 
use  a  variation  on  Grano vetter’s  work  on  identifying  strong  and  weak  ties  (Grano vetter 
1973)(Poran  and  Sabastien  1998).  Using  this  framework  will  help  determine  the  major 
contributors  to  the  output  of  the  system  under  study.  The  assignment  of  weak  v.  strong 
relationships  between  elements  in  a  system  such  as  TNT  is  important  when  establishing 
feedback  loops.  In  applying  Granovetter’s  methods  to  this  application  we  are  proposing  that 
strongly  tied  elements  rely  on  each  other  for  information  essential  to  mission  success. 

Weakly  tied  elements  either  provide  non-mission  essential  information  to  each  other  or  have 
multiple  weak  ties  to  other  elements  that  can  provide  the  essential  information  (i.e., 
redundant  links.) 

System  Environment 

The  purpose  of  the  LLNL/TacSAT  network  is  to  assist  the  US  Coast  Guard  in  positively 
identifying  contraband  radiological  material  in  ships  and  watercraft  entering  US  waters.  The 
expansiveness  of  US  controlled  waters  and  the  jurisdiction  of  the  US  Coast  Guard  make  it 
unfeasible  for  radiological  technical  experts  to  be  on-scene  for  every  inspection,  therefore  it 
is  essential  that  the  information  be  brought  to  the  expert  for  evaluation.  Networked  systems 
allow  data  acquired  by  the  boarding  party  to  be  delivered  to  a  geographically  dispersed  set  of 
experts  to  ensure  the  on-scene-commander  has  the  right  information  to  make  the  correct 


12 


Distributed  Holistic  Bounding 


decision  and  minimize  the  possibility  of  a  type  II  error4.  Figure  1  depicts  the  basic 
architecture  of  the  LLNL/TacSAT  network  that  supports  this  scenario. 


Figure  1:  LLNL/TacSAT  Topology 


4  In  this  case  the  alternate  hypothesis  for  the  boarding  party  is  that  the  ship  is  carrying  contraband  radiological 
material,  therefore  the  null  hypothesis  is  that  the  ship  is  carrying  no  such  material.  The  significant  impact  of 
unauthorized  radiological  material  entering  the  US  is  such  that  the  probability  of  a  type  II  error  (accepting  a  false 
null  hypothesis  as  true)  must  be  low  (Kerlinger  2000). 
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The  Tactical  Network  Topology 

The  TNT  is  an  evolving  network  designed  to  increase  the  efficiency  and  effectiveness  of  US 
armed  forces  through  exploitation  of  multi-path  transmissions  and  mesh  network  operations. 
The  TNT  is  not  a  single  system  but  rather  a  concept  of  integrating  many  network 
technologies  including  desktop  and  laptop  computers,  handheld  Personal  Data  Assistants 
(PDAs),  and  mobile  relay  devices  (Bordetsky  2005).  As  previously  discussed,  the  purpose  of 
the  GIG  is  to  obtain  and  maintain  information  superiority  over  an  adversary,  the  TNT,  as  a 
part  of  the  GIG,  must  somehow  contribute  to  this  objective.  Identification  of  the  system’s 
objective  (or  reason  for  being)  is  significant  to  identifying  the  appropriate  feedback 
mechanisms  necessary  to  help  control  the  system  in  order  to  meet  this  objective.  The  TNT 
supports  the  GIG’s  overall  objective  by  helping  ensure  that  the  right  information  can  get  to 
the  right  people  at  the  right  time. 

For  the  TNT  to  meet  its  goal  there  must  be  a  mechanism  that  can  monitor  the  information 
processes  and  make  alterations  or  allow  alterations  to  be  made  should  outputs  fall  outside  of 
acceptable  parameters  (Franklin  1994)  By  measuring  the  actual  output  of  the  system  against 
the  intended  output  we  can  identify  possible  problems  within  the  system  and  more 
adequately  troubleshoot  these  issues  before  they  result  in  catastrophic  failure  of  the  system. 
The  intent  of  the  TNT  and  its  multi-path,  multi-platform  architecture  is  to  allow  information 
to  flow  despite  individual  system  failures.  The  task  now  is  to  identify  and  describe  the 
elements  that  comprise  the  feedback  process  and  develop  measures  and  means  to  help  control 
the  system  based  on  the  feedback  comparisons  (Johnson  1962).  Identifying  system  elements 
begins  with  locating  the  boundaries  of  the  system  under  study. 
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The  TNT  exists  to  support  armed  forces  personnel  in  myriad  missions  in  the  battlespace  by 
providing  robust  communication  links  between  dispersed  elements.  Although  there  are  a 
considerable  number  of  missions  that  might  use  the  TNT  all  of  them  rely  on  the  same 
fundamental  command  and  control  concepts  to  accomplish  the  objective.  Command  and 
Control  is  the  phenomenon  through  which  a  commander  with  appropriate  authority 
commands  troops  to  accomplish  a  specific  mission,  this  is  done  through  personnel, 
equipment  and  procedures  (U.S  Department  of  the  Navy  1995).  This  study  focuses  on  the 
equipment  portion  of  command  and  control.  By  limiting  our  investigation  to  one  area  of 
command  and  control  we  can  explore  a  single  mission  associated  with  the  TNT  as  a  model 
for  future  assessment  of  other  mission  areas  or  uses  of  the  TNT.  This  paper  focuses  on  the 
desired  outcome  of  identifying  contraband  radioactive  material  on  board  sea  going  vessels. 

The  process  of  identifying  such  material  involves  the  US  Coast  Guard  intercepting  and 
boarding  a  vessel  suspected  of  carrying  such  material  (target  ship)  and  dispatching  a 
boarding  party  to  inspect  the  vessel  for  contraband.  Identification  of  radioactive  material  is 
outside  of  the  expertise  of  US  Coast  Guard  boarding  parties  so  a  means  of  remote  rapid 
identification  must  be  available  for  the  Coast  Guard  to  meet  its  objectives  (Klopson  2005). 
The  TNT  architecture  provides  a  possible  solution  to  this  problem. 

Lawrence  Livermore  Laboratories  has  developed  a  portable  radiation  detection  device  that 
produces  a  digital  output  for  computer  analysis.  Using  elements  of  the  GIG,  the  boarding 
party  can  gather  data  onboard  the  target  ship  and  transmit  these  data  (via  TNT  and  the  GIG) 
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to  radiation  experts  located  at  laboratories  ashore  for  near-real  time  evaluation.  These 
evaluations  can  then  be  transmitted  back  to  the  boarding  party  where  appropriate  action  can 
be  taken  should  the  material  indeed  by  contraband. 

Without  a  proper  feedback  mechanism  between  the  laboratory  personnel  and  the  boarding 
party  there  would  be  no  assurance  that  the  proper  data  were  transmitted  or  received.  This 
could  lead  the  boarding  party’s  not  taking  appropriate  actions  based  on  misinterpretation  or 
incomplete  transmission  of  information.  (Klopson,  2005) 

Determining  Relationships  Among  System  Elements 

Examining  the  scenario  with  Coast  Guard  personnel  revealed  that  there  are  five  major 
elements  that  must  be  present  in  this  scenario  (Klopson  2005): 

•  Target  Ship  (a  vessel  suspected  of  carrying  contraband  material) 

•  Launch  Ship  (a  vessel  capable  of  intercepting  the  target  ship) 

•  Boarding  Party  (a  group  of  individuals  from  the  Launch  Ship  capable  of  boarding  and 
inspecting  the  target  ship) 

•  Detection  Equipment  (equipment  or  personnel  capable  of  determining  the  location 
and  nature  of  contraband) 

•  Expert  personnel  (remotely  located  at  various  locations  throughout  the  US) 
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Examining  the  relationships  between  these  elements  should  reveal  whether  they  are  strongly 
or  weakly  tied.  This  definition  of  strong  ties  differs  from  Granovetter’s  in  that  he  identifies 
strong  ties  with  temporal  and  emotional  commitments  between  elements  (Granovetter  1973) 
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and  here  we  identify  strong  ties  by  assessing  informational  commitment  between  elements. 
Therefore,  care  should  be  taken  to  assess  the  number  of  unique  information  sources  when 
determining  the  strength  of  ties,  and  not  simply  the  number  or  proximity  of  links. 

The  portion  of  TNT  under  study  exhibits  an  overall  strong  tie  between  the  detection 
equipment  and  the  expert  personnel  because  each  is  solely  dependent  on  the  other  for  the 
information  necessary  to  complete  their  mission.  This  occurs  because  the  information 
necessary  to  make  the  assessment  (of  whether  specific  material  is  contraband)  is  resident  on 
the  target  ship  and  gathered  by  the  boarding  party.  There  are  also  strong  ties  between  the 
boarding  party-launch  ship  and  the  launch  ship-expert  personnel  because  the  boarding  party 
represents  the  only  source  for  the  vital  data.  Should  the  technology  exist  that  would  allow 
the  boarding  party  to  communicate  directly  with  the  expert  personnel  the  launch  would 
become  a  weak  tie  in  this  scenario. 


Identifying  the  Feedback  Loop 

The  existence  of  strong  ties  indicates  a  tenuous  relationship  between  elements;  relationships 
that  must  be  monitored  and  controlled  to  ensure  the  links  are  maintained.  This  control  is 
accomplished  through  a  feedback  loop.  Earlier  we  established  that  a  feedback  loop  must 
consist  of  the  following  elements  (Franklin  and  Lee  1994): 

•  Reference  (desired  outcome) 

•  Output  sensors  (to  measure  actual  outcome) 

•  Comparator  (which  compares  output  data  with  input  data) 
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•  Process  (for  doing  useful  work  with  the  compared  data) 

•  Actuator  (which  influences  the  input  variable) 

Therefore,  we  must  identify  the  elements  of  the  TNT  system  that  perform  these  functions. 

•  Reference:  positive  identification  of  suspect  material 

•  Output  sensors:  detection  equipment 

•  Comparator:  expert  personnel 

•  Process:  maneuvering  detection  equipment  into  position  to  acquire  data 

•  Actuator:  Coast  Guard  personnel  operating  the  detection  equipment 

The  feedback  loop  in  this  case  would  function  to  help  the  expert  personnel  positively  identify 
whether  the  material  in  question  is  or  is  not  contraband  by  allowing  them  to  receive  data  on 
the  material  and  direct  the  boarding  party  to  gather  more  or  different  data  to  ensure  accurate 
assessment  of  the  material.  Now  we  can  evaluate  this  cycle  to  determine  what  parameters 
and  criteria  can  be  measured  to  assess  the  effectiveness  of  the  feedback  process. 

Feedback  Parameters  and  Criteria 

The  objective  of  the  system  is  to  provide  adequate  data  to  the  expert  personnel  for 
assessment  of  suspect  material.  As  identified  earlier,  there  are  three  parameters  that  can  be 
used  to  assess  the  effectiveness  of  system  achieving  Information  Superiority:  completeness, 
correctness  and  currency.  The  data  not  only  have  to  be  adequately  collected,  they  must  be 
transmitted  to  experts  for  analysis  ant  the  experts  must  be  able  to  communicate  with  the 
boarding  party  about  the  quality  of  the  data  and  also  their  findings  (e.g.,  if  the  data  are 
incomplete  or  insufficient  for  analysis  the  boarding  party  may  have  to  acquire  more  data,  or 
if  contraband  the  Coast  Guard  must  take  appropriate  action,  etc.)  This  requires  identification 
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of  the  design  variables  within  the  criteria  of  interest  (denoted  as  cci,....,  ccn)  and  associated 
performance  criteria  constraints  (denoted  as  phi  (<|))): 

•  (Xi  =  Completeness/Data  Integrity 

o  <t>n  =  Percentage  of  target  ship  accessible  by  the  sensor 
o  <j>i2  =  Percentage  of  the  accessible  area  actually  scanned  by  the  sensor 
o  <(>i3  =  Aggregate  packet  loss  between  sensor  and  laboratory  expert 
o  <(>i4  =  Error  rate 

•  a2  =  Correctness.  In  this  case  correctness  is  the  degree  to  which  the  recorded  data  (at 
the  sensor)  approximates  the  actual  “ground  truth”  of  the  contraband  material  (Perry, 
Signori,  Boon  2004).  For  the  purposes  of  these  experiments  this  is  held  constant  (i.e., 
a2=  1) 

•  CX3  =  Currency/Time 

o  <(>31  =Time  from  acquisition  of  data  (via  the  detector)  to  time  of  receipt  by  expert 
personnel 

o  ((>32  =Total  time  for  evaluation  by  expert  personnel 

o  ((>33  =Time  from  determining  findings  by  experts  to  time  of  receipt  by  boarding 
party 

o  <J>34  =  Data  transfer  rate  between  detection  equipment  and  boarding  party  laptop 
computer 

o  <j>35  =  Data  transfer  rate  between  the  laptop  computer  and  the  launch  ship 
relay/bridge  (Rc_s) 
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o  (j>36  =  Data  transfer  rate  between  the  ship  and  the  expert  analyst 

Each  of  these  criteria  can  be  applied  to  each  node/link  in  the  system,  however,  a  feedback 
loop  is  designed  to  adjust  a  process  based  on  deviation  of  an  output  from  the  desired 
outcome.  Therefore,  the  parameters  associated  with  these  criteria  should  be  measured 
between  the  input  (detection  equipment)  and  the  output  (completion  of  expert  analysis  on 
received  data)  and  then  between  the  expert  analysts’s  response  to  the  boarding  party  and  the 
boarding  party’s  actions  based  on  that  response.  The  objective  of  this  system  is  to  provide 
maximum  confidence  in  data  analysis  while  minimizing  the  time  it  takes  to  properly  identify 
the  suspect  material  to  the  boarding  party,  therefore  we  can  develop  functions  associated  with 
measuring  system  effectiveness;  the  objective  function  would  be  one  that  will  maximize  oq 
with  the  minimum  0C3.  The  Pareto  set  for  inclusion  in  a  Parameter  Space  Investigation  (PSI) 
model  becomes  (Statnikov  2005): 

•  Data  Completeness  (oq)  is  a  function  of  packet  loss  (((>13)  and  error  rate  (E); 

oq=f((|)i3,  <j>i4) 

•  Time  between  data  acquisition  (<|>3i)  and  the  conclusion  of  expert  analysis  (<(>32)  is 

OC3  =  f(^31,  <t>32,  Keq,  <j>34,  <|)35,  <j>36) 

Although  further  study  and  development  are  needed  to  develop  the  associated  equations  and 
the  means  to  measure  (X2,  the  existence  of  numerous  measurable  design  variables  within  these 
two  criteria  (data  completeness  and  currency)  indicate  that  this  feedback  system  can  be 
quantitatively  evaluated  to  develop  an  optimal  design.  The  multi- variable  nature  of  this 
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problem  indicate  that  the  Dr.  Roman  Statnikov’s  Parameter  Space  Investigation  (PSI)  method 
of  multi-criteria  optimization  may  be  the  best  approach  to  identifying  optimal  conditions  for 
controlling  systems  within  the  TNT. 

System  Summary 

This  paper  identified  a  means  of  evaluating  a  feedback  mechanism  within  one  element  of  the 
GIG.  It  demonstrated  that  the  criteria  by  which  the  effectiveness  of  that  mechanism  can  be 
identified  and  measured  through  analysis  and  evaluation  of  the  ties  between  major  elements 
of  the  system.  The  parameters  associated  with  these  criteria  were  also  identified  and 
objective  functions  were  outlined  as  a  launch  point  of  further  investigation  through  the  PSI 
method  of  multi-criteria  optimization.  Application  of  this  model  should  provide  system  users 
with  near-real  time  assessments  of  the  effectiveness  of  the  feedback  loop  within  this 
particular  TNT  functional  area.  By  understanding  the  relationship  between  data 
completeness  and  timeliness  the  system  users  can  assess  whether  adjustments  must  be  made 
to  the  network  in  order  to  meet  mission  objectives.  This  foundation  will  allow  development 
of  experiments  that  can  explore  the  viability  of  the  LLNL/TacSAT  and  TNT  networks  in 
various  environmental  conditions. 

Experiment  Design 

We  have  explored  the  purpose  and  basic  functionality  of  the  LLNL/TacSAT  network  and 
determined  that  the  system  must  be  capable  of  gathering  and  delivering  data  in  a  manner  that 
will  allow  US  Coast  Guard  boarding  parties  to  make  accurate  assessments  of  the  existence  or 
non-existence  of  contraband  radiological  material  on  board  specific  vessels.  At  issue  now  is 
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how  to  gather  information  that  will  help  us  construct  a  system  that  will  accomplish  this 
objective.  We  must  design  an  adequate  experiment  campaign  strategy  that  will  provide 
verifiable  evidence  that  the  LLNL/TacSAT  system  can  meet  its  design  objectives  in 
anticipated  operational  environments.  Observation  of  the  system’s  performance  within  a 
controlled  environment  is  essential  to  this  type  of  scientific  exploration  (Cohen  and  Nagel 
1934). 

As  discussed,  the  GIG  and  the  LLNL/TacSAT  networks  are  large  information  systems 
spanning  several  topologies  and  geographic  regions.  As  such  there  will  be  a  considerably 
large  number  of  variables  that  will/could  affect  the  system  and  hence  any  experiments 
attempting  to  ‘grab’  all  of  these  variables  in  one  experiment.  Therefore  it  is  necessary  to 
conduct  a  series  of  experiments  that  systematically  considers  sets  of  these  variables  until  the 
most  favorable  system  can  be  developed.  This  systematic  approach  can  be  considered  an 
experiment  campaign  (Alberts  and  Hayes  n.d.) 

The  experiment  campaign  begins  with  the  recognition  of  a  problem  and  developing 
expressions  that  put  this  problem  into  a  manageable  form  (Kerlinger  and  Lee  2000).  This 
may  be  the  most  critical  part  of  the  campaign,  and  perhaps  the  most  difficult.  Without 
adequately  identifying  the  problem  or  the  question  to  be  answered,  experiments  cannot  be 
designed  that  will  reveal  useful  information  (Kerlinger  and  Lee  2000.)  Although  a  viable 
system  may  result,  there  is  no  means  of  determining  if  the  system  is  optimal  or  that  is  can  be 
expected  to  properly  function  in  various  environments.  This  first  step  of  the  experiment 
campaign  can  be  accomplished  through  discovery  experiments  (Alberts  and  Hayes  n.d.). 
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Following  the  formulation  of  an  acceptable  problem  statement  the  campaign  will  progress  to 
hypothesis  development  and  testing.  A  hypothesis  is  a  conjectural  statement  about  the 
relationship  between  two  or  more  phenomena  or  variables  (Kerlinger  and  Lee  2000:  15). 
Once  a  hypothesis  has  been  put  forth  concerning  the  problem  identified  in  the  discovery 
experiments  the  campaign  can  proceed  to  test  these  hypotheses.  Alberts  and  Hayes  divide 
hypothesis  testing  into  two  phases:  preliminary  and  refined  hypothesis  testing  (Alberts  and 
Hayes  n.d.).  The  goal  of  preliminary  hypothesis  testing  is  to  develop  a  basic  evidence  base 
concerning  the  problem  and  related  hypotheses.  The  refined  tests  hope  to  develop  predictive 
knowledge  about  the  problem.  The  experiment  campaign  concludes  with  a  demonstration  of 
what  was  learned.  Alberts  and  Hayes  dubbed  these  demonstration  experiments.  This 
experiment  campaign  methodology  can  be  applied  to  our  investigations  of  the 
LLNL/TacSAT  system  and  should  provide  a  bounded  means  of  developing  an  optimal 
system  within  the  GIG  to  help  achieve  Information  Superiority. 

The  LLNL/TacSAT  Experiment  Campaign 

The  first  step  of  the  experiment  campaign  was  described  earlier  in  this  paper  where  we 
identified  a  problem:  that  of  providing  correct,  complete,  and  current  expert  assessment  of 
potentially  contraband  material  to  a  remotely  located  US  Coast  Guard  boarding  team.  At 
issue  is  how  to  best  accomplish  this.  Although  the  LLNL/TacSAT  network,  as  designed, 
consists  of  proven  technology  (see  figure  1)  and  there  have  been  some  preliminary 
technology  demonstrations  (Klopson  2005)  there  has  been  little  experimentation  designed  to 
discover  the  environmental  variables  that  could  affect  system  performance.  Rather  that  leave 
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it  to  chance  that  the  system,  as  designed,  will  work  in  varying  operational  environments  an 
experiment  campaign  beginning  with  extensive  discovery  experimentation  is  proposed. 

As  identified  earlier,  we  have  three  areas  of  interest  for  measuring  information  related  to 
obtaining  Information  Superiority:  completeness,  correctness  and  currency.  By  conducting 
discovery  experiments  associated  with  each  of  these  areas  and  determining  the  variables  or 
phenomena  associated  with  each  we  will  be  better  poised  to  develop  (and  test)  hypotheses 
focused  on  creating  an  optimal  system.  This  group  of  discovery  experiments  can  be 
considered  a  campaign  in  and  of  itself  within  the  framework  of  the  overall  experiment 
campaign.  Each  individual  discovery  experiment  will  consist  of  several  ‘engagement’ 
experiments.  An  engagement  experiment  is  one  which  focuses  on  providing  evidence 
associated  with  a  single  phenomenon  within  its  overarching  experiment  campaign  (i.e.,  an 
experiment  campaign  would  consist  of  discovery,  preliminary  and  refined  hypothesis,  and 
demonstration  campaigns  (Alberts  and  Hayes  n.d.)  and  each  of  these  would  consist  of 
discovery  engagements,  preliminary  hypothesis  engagements,  etc.))  Here  we  will  develop 
the  framework  for  a  discovery  experiment  focused  on  the  completeness  of  measured  data. 

In  our  case  the  discovery  experiment  campaign  is  designed  to  identify  the  most  significant 
variables  (environmental,  organizational,  etc.)  that  might  affect  system  performance  (as 
related  to  mission  objectives.)  As  in  any  experiment  the  most  important  factors  are 
identifying  and  bounding  the  problem  (and  adequately  articulating  the  problem.)  This 
applies  to  engagement  experiments  as  well.  The  remainder  of  this  paper  will  outline  the 
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process  for  developing  an  initial  ‘discovery  engagement’  focused  on  identifying  variables 
affecting  completeness  of  information  within  the  system. 

As  shown  in  figure  1  there  are  several  links  between  the  initial  data  acquisition  point  (the 
LLNL  radiological  sensor)  and  the  expert  analysts  ashore  (in  this  case  they  are  assumed  to  be 
located  at  the  NPS  NOC.)  Identifying  variables  that  could  affect  signal  completeness  along 
this  path  can  be  done  through  brainstorming  sessions  with  information  systems  experts.  This 
method  will  help  develop  an  initial  list  of  factors  that  have  been  known  to  cause  signal 
disruption,  interference,  or  any  adverse  affects  for  each  of  the  transmission  methods  and 
media  (e.g.,  802.1 1,  802.16,  UHF,  copper,  etc.)  in  various  environmental  or  physical 
conditions  (e.g.,  rain  rate,  sunspots,  sea  state,  ship  material,  foliage,  etc.)  After  developing  a 
baseline  listing  of  possible  variables  further  experiment  design  can  proceed. 

The  next  step  is  to  develop  a  list  of  engagement  experiments  that  will  adequately  explore  the 
possible  relationships  of  these  variables  to  the  different  links  of  the  system.  This  will  help 
identify  what  variables  have  debilitating  affects  on  system  performance  (associated  with  the 
criterion  of  interest:  completeness)  and  provide  evidence  for  future  study  on  mitigating  the 
effects  of  these  variables.  By  developing  experiments  and  models  that  account  for  the 
variables  associated  with  one  link  of  the  system,  other  experiments  within  the  overall 
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campaign  can  draw  on  these  existing  models  and  perhaps  treat  each  engagement  scenario  as  a 
single  variable  within  this  new  engagement  scenario.5 

When  assessing  the  flow  of  data  from  the  LLNL  sensor  to  the  expert  in  the  NPS  NOC  it  is 
useful  to  use  basic  electronic  troubleshooting  techniques  (Buban  and  Schmitt  1972). 
Troubleshooting  involves  the  location  of  correction  of  any  condition  adversely  affecting  a 
systems  performance.  This  requires  knowledge  of  the  input  data,  knowledge  of  the  expected 
output  data  and  knowledge  of  the  components  and  process  that  act  on  the  data  between  the 
input  and  output.  This  is  why  the  assessments  and  system  bounding  discussed  at  the 
beginning  of  this  paper  are  so  important.  By  understanding  the  system  boundaries  and  the 
expected  functionality  within  these  boundaries  we  can  better  understanding  the  effects  each 
variable  will  have  on  completeness  of  our  data. 

This  method  of  bounding  the  experiment  helps  control  the  independent  variables  in  the 
experiment  and  account  for  any  extraneous  variables  that  may  affect  the  outcome  but  not  be  a 
part  of  the  study.  The  fidelity  of  the  experiment  (i.e.,  the  assurance  that  the  results  will 
translate  to  ‘real-world’  systems)  is  maintained  in  this  instance  because  the  experiment  is 
conducted  with  actual  operational  systems.  There  are  aspects  of  this  experiment,  however, 
that  can  be  replicated  in  a  laboratory  or  incorporated  from  previous  research  (e.g.,  the 
propagation  characteristics  of  wireless  transmissions  through  various  materials,  etc.)  Care 
must  be  taken,  however,  to  account  for  actual  conditions  that  may  exist  in  the  expected 

5  This  method  of  treating  a  single  engagement  experiment  as  a  single  variable  (or  object)  within  another  engagement 

experiment  is  not  fully  developed,  however,  I  will  identify  this  method  of  experimentation  as  Object  Oriented 

Experimentation  (OOE). 
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operating  environment  (e.g.,  salinity,  corrosion,  paint,  etc.)  Again,  expert  brainstorming 
sessions  prior  to  model  development  will  increase  the  fidelity  of  such  methods. 

Upon  completion  of  this  engagement  experiment  the  investigator  should  have  a 
comprehensive  (and  well  documented)  understanding  of  the  variables  within  the  operational 
environment  that  will  impact  data  completeness.  From  these  results  the  overall  campaign  can 
progress  into  the  preliminary  hypothesis  development  and  testing  phase.  Development  of 
this  phase  lies  outside  the  scope  of  this  paper,  however,  it  is  in  this  phase  where  ideas  can  be 
developed  and  explored  on  means  to  mitigate  the  possible  adverse  affects  identified  in  the 
discovery  campaign.  Each  of  these  hypotheses  can  be  explored  in  individual  engagement 
experiments  utilizing  the  Object  Experiments  developed  in  previous  engagement 
experiments.  Alberts  and  Hayes  provide  a  comprehensive  checklist  of  items  for  developing 
an  experiment  campaign  and  individual  experiments  (Alberts  and  Hayes  n.d.). 

Summation 

Testing,  evaluating  and  analyzing  large,  scale-free  information  systems  are  possible  through 
distributed  holistic  bounding  and  employment  of  disciplined  experiment  campaigning. 
Beginning  with  identification  of  the  overarching  purpose  of  the  entire  system  and  identifying 
the  boundaries  of  the  sub-systems  that  contribute  to  the  overall  system’s  functionality  we  can 
develop  effective  experiments  that  will  identify  critical  environmental  variables  that  affect 
system  performance.  By  adhering  to  an  experiment  campaign  strategy  we  can  discover  those 
variables,  develop  hypotheses  that  help  determine  optimal  system  configurations  and 
ultimately  build  and  demonstrate  that  optimal  system. 
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Problem 


Analyzing  the  performance  of  large, 
scale-free  systems 

-  Changing  boundaries 

-  Changing  components 

-  Nebulous  system  objectives  (Information 
Superiority) 


The  GiG  &  Information  Superiority 


Global  Information  Grid  (GIG):  a  large,  scale 
free  system  “designed”  to  meet  a  specific 
objective  (Information  Superiority) 

The  GiG  consists  of  a  constantly  changing 
number  of  nodes  and  links 

Information  Superiority  is  a  measure  of  one’s 
own  information  capabilities  against  those  of 
an  enemy  (another  constantly  changing  area) 


Feedback 


Accomplishing  effective  system  analysis  against 
stated  objectives  is  irrelevant  without  a  means  of 
correcting  poor  system  behavior  (i.e.,  behavior  that 
does  not  produce  desired  outcomes) 

Feedback  must  be  considered  a  part  of  the  system 
under  study 


Bounding  the  Problem 


The  GiG,  by  definition,  is  a  global  system;  much  too 
large  to  analyze  as  a  whole 

A  concept  should  be  developed  for  parsing  the  GiG 
into  adequately  small  segments  for  study,  but 
adequately  large  segments  to  still  have  meaning  in 
assessing  Information  Superiority 


Parsing  the  GiG 


Through  understanding  stakeholder  influences 
(Rowley  1977)  and  network  cliques  (Provan  and 
Sebastian  1998)  we  can  identify  “unique”  systems  within 
the  GiG  that  have  specific  network  roles  in  gaining 
Information  Superiority 

Studying  the  nature  of  stakeholders  and  cliques  and  the 
strength  of  the  ties  between  them  (Granovetter  1973) 
gives  us  a  layout  of  information  systems  within  the  GiG 
that  contribute  to  overall  system  performance  for  specific 
operations 


System  Identification 


Specific,  desired  outcome 

-  Contraband  identification  on  inbound  shipping 

-  Lawrence  Livermore  National  Labs/TacSAT 

Stakeholders  in  process 

-  Boarding  Party 

-  USCG  Commander 

-  Expert  Personnel 

Stakeholder  information  needs 

Feedback  mechanisms  within  the  system 

System  success  criteria  (determined  by  the  on-scene 
commander) 


Bound  the  system 


Based  on  the  commanders  criteria  for  success 

Components  affecting  information  critical  to  these 
criteria  are  considered  within  the  system  (strongly 
tied  to  the  outcome) 

Components  affecting  information  that  is  ‘nice-to- 
have’  should  be  considered  outside  the  system 
(loosely  tied  to  the  outcome) 


Distributed  Holistic  Bounding 


This  method  of  bounding  the  LLNL/TacSAT  system  strives  to 
treat  this  system  holistically  (i.e.,  identifying  information  and 
systems  that  are  critical  or  non-critical  its  functionality.  From 
here  the  system  can  be  assessed/managed  based  on  the 
potential  debilitating  effect  of  the  environmental  changes  within 
and  without  of  the  system) 

Further  assessment  of  systems  and  information  that  affect  the 
performance  (but  not  critically)  of  the  LLNL/TacSAT  should 
reveal  the  other  systems  that  contribute  to  the  performance  of 
the  LLNL/TacSAT  but  have  their  own  critical  function  within  the 
GiG. 
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By  treating  each  of  the  strongly  tied  systems  as 
holistic  entities  linked  to  each  other  through  weak 
ties  we  can  begin  to  develop  models  that  help 
determine  how  well  a  large,  scale-free  system  (such 
as  the  GiG)  is  meeting  its  objectives  (information 
superiority)  and  aid  in  identifying  areas  where 
efficiency  or  effectiveness  may  not  be  optimal. 

I  call  this  Distributed  Holistic  Bounding 


